Attorney Docket 22I4P 

UNITED STATES PATENT APPLICATION 

FOR 

OPERAND FILE USING POINTERS AND REFERENCE 
COUNTERS AND A METHOD OF USE 



Inventor: 
Seungyoon Peter Song 



Sawyer Law Group LLP 

2465 E. Bayshore Road, Suite 406 

Palo Alto, California 94303 



OPERAND FILE USING POINTERS AND REFERENCE COUNTERS 

AND A METHOD OF USE 

FIELD OF THE INVENTION 

The present invention relates generally to a processing system and specifically to an 
operand file utilized within tiie processing system. 

BACKGROUND OF THE INVENTION 

Almost all processors are designed to operate in a pipeline; the simplest of which 
consists of the fetch, decode, and execute stages. Instructions are fetched (or read) firom 
memory in the fetch stage. They are then decoded to determine what operations to perform 
on which operands in the decode stage. The actual operations are performed in the execute 
stage. Most high-performance processors use additional pipeline stages to increase the 
operating speed or the number of instructions that can be processed simultaneously (in one 
clock cycle) or to speculatively process instructions before it is known that these instructions 
are to be processed at all. 

The results of executing instructions are stored in registers or in memory. The results 
that are used immediately or repeatedly are generally kept in registers, since registers can be 
accessed much faster dian memory. The registers can be implemented using individual flip- 
flops or latches but are generally implemented using SRAM, known as a register file, to 
minimize the area occupied by the registers. A 32-bit processor with 16 general-purpose 
registers, for example, would use a register file consisting of SRAM organized as 16 words 
of at least 32 bits per word. A register file is designed to support multiple read and write 
operations per clock cycle. For instance, a register file may support four read and two write 
operations to sustain execution of two instructions in each cycle, assuming that the 
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instoictions use two operands and produce one result. Such a register file is said to have 
four read ports and two write ports. Processors may also have special-purpose registers that 
serve specific functions, such as keeping processor control and status information, providing 
debug or performance monitoring information, or aid in translating from virtual address to 
physical address. Although special-purpose registers may be better implemented as 
individual flip-flops and general-purpose registers in a register file, the same set of rules 
apply to reading and writing either type of registers, as described below. 

If an instruction is executed before all instructions that are earlier in the program 
sequence have executed, its results must not be written to the specified register or memory if 
the processor is to provide a programming model known as precise exception. Such 
behavior is required when an earlier instruction produces an error condition, in which case 
the results of this "prematurely executed" instruction must be discarded without affecting 
any of the processor's registers or memory. To be exact, the processor must behave as if it 
executed all instructions that are earlier than the one causing the error and none of the 
instructions that are later than the one causing the error. The result of any prematurely 
executed instructions must, therefore, be kept in temporary storage. 

Many processors use a rename buffer to hold these temporary results until it is safe to 
update the intended destination registers or memory with the results. The rename buffer is 
said to hold the fiiture states - as opposed to the architectural state - because it contains the 
results that may or may not be updated to their intended destination registers or memory. As 
each instruction is executed in tiie program sequence and does not cause an error, its results 
can be safely and permanently copied to its specified memory or destination registers. Such 
an mstruction is said to be completed and its destination registers are said to hold tiie 
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architectural state. If an instruction causes an error, its results as well as the results of any 
prematurely executed instructions in the rename buffer are discarded. 

Many high-perfonnance processors execute a later instruction before executing an 
earlier one if the later instruction is ready to execute while the earlier one is not. They 
generally use an additional pipeline stage between the stages where the source operands are 
read and the instructions are executed. They use a reservation station to hold the instructions 
in this intermediate stage. As an instruction enters the reservation station, it obtains the 
source operands from the instruction itself for immediate operands or from memory, the 
register file or the rename buffer for register operands. If a source operand is not yet valid in 
memory, the register file or tiie rename buffer, it must be the destination of an earlier 
instruction that has not yet executed. When this earlier instruction is executed, its results are 
written to the rename buffer (assuming that all results are first written to the rename buffer 
before they are copied to memory or the register file) and to the source operand fields of the 
waiting instructions in the reservation station. The latter process is known as result 
forwarding, which allows the waiting instructions to obtain the source operands without 
reading memory, the rename buffer or register file. 

Rename buffer is one of many names that refer to the storage elements used to hold 
fiiture results until the results can be safely and permanently written to their intended 
destination registers or memory. Reservation station is also one of many names tiiat refer to 
the storage elements used to hold flie source operands of instructions waiting to be executed. 

The advantage of operand file is that it eliminates copying results and operands 
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between the register file, reservation station, and rename buffer, thereby greatly shnplifying 
the design and reducing area and power consumption. Furthermore, it can also be used in 
multithreaded processors that spawn children threads by copying some or all of the parent 
thread's registers to each of the children thread's registers. 

5 

SUMMARY OF THE INVENTION 

This disclosure describes an operand file, a device that combines the functions of a 
register file, a reservation station, and a rename buffer into single storage element. The 
Ip- : advantage of this mechanism is that it eliminates copying results and operands between the 

3J register file, reservation station, and rename buffer, thereby greatly simplifying the design 
and reducmg area and power consumption. Furthermore, it can also be used in 
s multithreaded processors tiiat spawn children threads by copying some or all of the parent 

thread's registers to each of the children thread's registers. 

if 

& V 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 illustrates an operand file in accordance with the present invention. 
Figure 2 illustrates each of the first K operand queue entries being assigned to each of 
20 the K registers. 

Figure 3 illustrates if all K registers have undefined value upon reset, all K registers can 
be mapped to one, say the first, entry in the operand queue. 

th • • 

Figure 4 illxistrates assigning the K operand queue entry to destination register Rl by 
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writing K to Rl's future state pointer and incrementing the associated reference count. 

Figure 5 illustrates the reference count of entry being decremented by 1 when an 
instruction that updates Rl is completed, since entry was previously assigned to the 
destination register RL 

Figure 6 illustrates executing an instruction that copies Rl to RO by copying the 
operand queue entry number in Rl 's foture state pointer to the RO's future state pointer and 
incrementing tiie reference count of the associated operand queue entry, which is K. 

Figure 7 illustrates completing an instruction that copies Rl to RO by writing the new 
operand queue entry number, which is K, into the RO's architectural state pointer and 
decrementing the reference comit associated with the operand queue entry previously assigned 
toRO,whichisK-L 

Figure 8 illustrates the parent thread's RO and Rl referring to the K**^ entry while Rk-1 
refers to the 0 entry. 

Figure 9 illustrates copying the registers RO, Rl and Rk-1 from the parent thread to the 
corresponding registers in a child thread by copying the parent thread's architectural state 
pointers to the associated architectural state pointer of the child's thread and incrementing the 
associated reference counts. 

DETAILED DESCRIPTION 

The present invention relates generally to a processing system and specifically to an 
operand file utilized within the processing system. The following description is presented to 
enable one of ordinary skill in the art to make and use liie invention and is provided in the 
context of a patent application and its requirements. Various modifications to the preferred 
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embodiment and the generic principles and features described herein will be readily apparent to 
those skilled in the art. Thus, the present invention is not intended to be limited to the 
embodiment shown but is to be accorded the widest scope consistent with the principles and 
features described herein. 

This disclosure describes an operand file, a device that combmes the functions of a 
register file, a reservation station, and a rename buffer into single storage element. 

Description of Operand File 

Figure 1 illustrates an operand file 100 in accordance witii the present invention. The 
operand file 100 comprises K number of architectural state pointers 1 10, K number of future 
state pointers 120, an N-entry operand queue 130, and N number of reference counters 140. 
K is the number of registers that can have future values, meaning instructions can 
prematurely execute and produce future values for these registers, and N is some number 
greater than K. Each of the K registers has an architectural state pointer and a corresponding 
future state pointer. The architectural state pointer identifies an entry in the operand queue 
that holds the architectural value of the register. That is, the entry has the result of the most 
recently completed instruction to modify the register. This instruction is not necessarily the 
most recently completed instruction. Similarly, the future state pointer identifies an entry in 
the operand queue that is assigned to hold the most recent future value of the register. That 
is, the entry is to hold the result of the most recently decoded instruction to modify the 
register. This instruction is not necessarily flie most recently decoded instruction. 
In summary, the operand file works as follows. When an instruction that modifies a 
destination register is decoded, a free operand queue entry is assigned to hold the future 
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value of the destination register by writing the free entry number into the register's future 
state pointer and incrementing the reference counter associated with the free entry by 1 . A 
free entry has the reference count of 0, indicating no register is referencing this entry. When 
there is not a free entry, the instruction is stalled until one becomes free. When this 
instruction is executed, its result is written to the operand queue entry assigned at decode. 
When the instruction is completed, the operand queue entry number is written to the 
destination register's architectural state pointer. The reference count of the entry that was 
previously assigned to the destination register (the entry number that was kept in the 
destination register's architectural state pointer before it was written with the new entry 
number) is decremented by l.In most cases, the decremented reference count becomes 0, 
indicating that the associated entry is now free. 

When an instruction that reads a source register is decoded, the entry number kept in 
the source register's ftiture state pointer is given to the instruction. When the instruction is 
ready to execute, it reads the source register's value from tiiis operand queue entry. When an 
instruction that copies one register to another (a common mnemonic for such an instruction 
is MOVE Rd, Rs), the operand queue entry number kept in the source register's ftiture state 
pointer is copied to the destination register's fixture state pointer. In addition, the reference 
count associated with the entry in the source register's fixture state pointer is incremented by 
1, since another register - the destination register of the move instruction - is referencing the 
entry. Using the operand file, a register copy instruction is effectively executed when the 
entry number in the source register's future state pointer is copied to the destination 
register's future state pointer and the entry's reference count is incremented. When the 
move mstruction is completed, the entry number in the source register's architectural state 
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pointer is copied to the destination register's architectural state pointer. In addition, the 
reference count associated with the entry that was previously assigned to the destination 
register is decremented by 1. 

The operand file 100 is now described in detail. The N-entry operand queue 130 can 
be implemented as a collection of flip-flops or latches. It is best implemented as a typical 
register file, however, having multiple read and write ports and being organized as N entries 
of W bits/entry. Instructions' source operands are read usmg the read ports and the 
instructions' results are written using the write ports. Any immediate operands that are 
specified within the instructions can also be written to the operand queue 1 30 - at any time 
before the instructions read the source operands from the operand queue 1 30 - so that they 
can be read back along with other source operands. The operand queue 130 should have 
more entries than the number of registers, since it can be used to hold immediate operands as 
well as the registers' future and architectural values. The width of the operand queue 130 is 
the same or larger than the widtii of the processor. That is, for a 32-bit processor, W is 32 or 
larger. 

The reference counter 140 is associated with each of the entries in the operand queue 
130. It indicates the total number of registers whose architectural or future value is kept in 
the associated operand queue entry. For instance, if the reference count for entry 2 is 3, there 
are total of 3 registers whose result is kept in entry 2. Obviously, all three registers must 
have the same value. The reference counter 140 may also indicate that the associated 
operand queue entry holds an immediate operand. When the reference count is 0, the 
associated entry is free since it is not being referenced by any register or holding an 
hnmediate operand. 
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Upon a reset, the architectural state pointers 1 10, the future state pointers 120, and 
the reference counters 140 are initialized using one of the two schemes. In the first scheme, 
each register is assigned its own operand queue entry. For example. Figure 2 illustrates each 
of the first K operand queue entries being assigned to each of the K registers. This requires 
each of the K architectural state pointers 1 10 to be initialized wifli unique operand queue 
entry number, ranging from 0 to K-1 . Each future state pointer 120 is initialized with the 
same value as its associated architectural state pointer. The reference count for the first K 
operand queue entries are set to 1, indicating that the associated entry is being referenced by 
one register. If a register has a defined value upon reset, the entry assigned to the register 
must be initialized with that defined value. In the second scheme, all registers that have 
undefined value upon reset are mapped to one (or more) operand queue entry and each of the 
remaining registers that have defined values upon reset is assigned its own operand queue 
entry. For example, Figure 3 illustrates if all K registers have undefined value upon reset, all 
K registers can be mapped to one, say the first, entry in the operand queue. Each of the 
architectural and future state pointers 1 10 and 120 would be initialized to 0, denoting the 
first entry in tiie operand queue, and the first entry's reference count would be initialized to 
K. The reference count of the remaining operand queue entries would be initialized to 0. 

When an instruction having a destination register is decoded, a free operand queue 
entry is identified and the entry number is written to the destination register's future state 
pointer 120. For example, if the instruction specifies Rl as a destination register, and the K^^ 
operand queue entry is free, the Rl 's future state pointer 120 is written with the entry 
number K and the reference count of the entry is incremented by 1 . This change is 
shown in Figure 4, using Figure 2 as the basis. When this instruction executes and produces 
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a result, its result is written to the K entry of the operand queue. When the instruction is 
completed, K is written to the Rl 's architectural state pointer while the reference count of 
the entry previously assigned to Rl is decremented by L That is, the reference count of K- 
2^^ entry is decremented by 1, as shown in Figure 5, since entry was previously 
assigned to Rl . There are two ways to determine which entry was previously assigned to a 
destination register. In one implementation, the destination register's architectural state 
pointer can be read before it is written with the new entry number. In another 
implementation, the destination register's fiiture state pomter can be read at decode and kept 
with the instruction until the instruction is completed. 

When an instruction having a source register is decoded, data for the source register 
is to be found at the operand queue entry indicated in the source register's future state 
pointer. This entry number is given to the instruction so that it can read the operand queue 
entry when it is ready to execute. For instance, if the instruction specifies RO as a source 

register, and the future state pointer for RO has K-1, then the data to be used for RO is located 

til 

at the K- 1 operand queue entry. 

When a register copy instruction is decoded, the instruction is actually executed by 
copying the entry number in the source register's future state pointer to the destination 
register's future state pointer and incrementing the entry's reference count by 1 . For 
example, when an instruction that copies from Rl to RO is decoded and Rl 's future state 
pointer has K, as shown in Figure 5, K would be written to the RO's future state pointer and 
the K* reference count would be incremented by 1, as shown in Figure 6. The reference 
count is now 2, since RO's and Rl 's ftiture state pointers are referencing it. Incidentally, 
Rl 's future state pointer is same as its architectural state pointer in the example, since there 
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is no pending insfruction that will modify Rl . When the register copy instruction is 
completed, K is written to RO's architectural state pointer and the K-1^ reference count is 

th 

decremented by 1, as shown in Figure 7, since K-r entry was previously assigned to the 
destination register RO before this register copy instruction was completed. 

If an instruction specifies an immediate operand, then the immediate data can also be 
written to a "firee" operand queue entry. For instance, when an instruction specifying an 
immediate operand is decoded and the K+1* operand queue entry is fi-ee, the K+1*** entry 
could be assigned to hold the immediate value. The reference count of the K+1* entry is 
incremented by 1 but the future or architectural state pointers are not changed. The K+1* 
entry number is given to the instruction so that it can read the immediate value from the 
operand queue along with any other source operands it may have. The immediate value can 
be written to the operand queue at anytime before the associated instruction reads it. 
Altematively, the associated instruction cannot read its immediate operand until the 
immediate operand is first written to tiie operand queue. At anytime after the immediate 
operand becomes no longer needed, the entry could be freed by decrementing the associated 
reference count. Li one implementation, the reference coxmt could be decremented as soon 
as the immediate operand is read from the operand queue, effectively freeing the entry. In 
another implementation, tiie reference count could be decremented when the associated 
instruction is completed. 

When an instruction to be completed next incurred an exception, the processor must 
stop completing instructions from the current instruction sequence and start executing 
instructions from the exception handler program. This requires adjusting the operand file so 
that it appears as if the instruction causing the exception has not yet altered it. Specifically, 
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the future state pointers and the reference counters must be restored to the values they had 
just before the mstruction causing the exception is decoded. In one implementation, each 
architectural state pointer is copied to its corresponding future state pointer and all pending 
instructions that must be discarded are designated "cancelled". There are many known ways 
to identify and designate instructions to be cancelled, and how this is done is irrelevant to the 
operation of the operand file, provided that all instructions, including those designated 
"cancelled", are processed by the instruction completion stage. When the completion stage 
processes an instruction to be cancelled, the destination register's architectural state pointer 
is not altered but the reference count associated with the entry assigned to the destination 
register is decremented by 1. 

Note that, if this instruction were to be completed and not cancelled, the reference 
count of the entry assigned previously to the destination would be decremented by 1 . If the 
instruction to be cancelled has an immediate operand, the entry assigned to hold the 
unmediate operand must also be decremented by 1. When all cancelled instructions are 
processed and no new instructions are decoded, the reference counts are restored to their 
values before any of the cancelled instructions were decoded. Once the future state pointers 
are restored in one cycle, however, instructions from the exception handler program can be 
decoded and executed while the cancelled instructions are being processed. 

The number of bits needed for the reference coimters can be kept to minimum while 
taking the full of advantage of the operand file's efficient register copying mechanism. In an 
implementation with K number of registers, it is possible for a reference count to become 
greater than K. For example, decoding K+1 number of register copy instructions, all 
specifyuig the same source register, could make the reference count to reach K+l before the 
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first such instruction is completed. However, such instruction sequence is useless and 
occurs extremely rarely. A preferred implementation is to use a few, say 2, bits for each 
reference counter. When a register copy instruction is decoded and the associated reference 
count is already at the maximum value, or 3 in this example, a free entry is assigned to hold 
the destination register's fixture value instead of copying the entry number from the source 
register's future state pointer. In this case, the instruction doesn't use the operand file's 
efficient register copy mechanism and, therefore, must be executed to physical copy the 
content of its source register to its destination register. The preferred implementation can be 
simplified further by disabling the operand file's register copy mechanism whenever any 
reference counter has reached its maximum. 

When a free entry is assigned to hold a register or an immediate value, it may be 
desurable to simply write the value of 1 to the associated reference counter instead of 
incrementing the reference count from 0 to 1, 

Incidentally, there are many ways to implement the reference counters, or counters in 
general. The most common approach, which is assumed in this disclosure, is to use natural 
numbers (numbers 0, 1 , 2, . . . and so on) and add 1 to increment and subtract 1 to decrement. 
One can also define an arbitrary counting sequence for the reference coimters and use a state 
machine to implement the counting sequence. For example, one can define 2-bit reference 
counters to count in the sequence of 2, 0, 3, and 1, with the value 2 denoting "free" entry. 

Using Operand File in Multithreaded Processors 

Multithreaded processors can be viewed as a tightly coupled set of single-threaded 
processors that share certain "tihread-independent" resources, such as multiply and floating- 
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point units or caches, in executing simultaneously related or unrelated sequence of 
instructions. The operand file is well suited to support multithreaded processors that 
dynamically create and termmate threads. The US Patent #5,812,81 1, entitled "Executing 
Speculative Parallel Instructions Threads with Forking and Inter-Thread Communication" 
and assigned to International Business Machines Corporation, describes such a processor in 
detail. 

The operand file efficiently supports creating a thread, which closely resembles what 
needs to happen upon a reset. The registers that have undefined value upon creation can all 
be assigned to one free entry in the operand queue by writing the entry's number into the 
registers' architectural and future pointers. The reference count of this entry is incremented 
by the number of registers with undefined values. Each of the registers that have defined 
value upon creation must be assigned a unique entry in the operand queue. How this is done 
is described previously. In some situations, some or all registers of a thread, known as the 
parent thread, are copied to the corresponding registers of a new thread being created, known 
as a child thread. The operand file efficiently supports this by copying the entry numbers 
kept in the parent thread's architectural (or fiiture) state pointers to the child thread's 
architectural (and future) state pointers and incrementing the associated reference counters, 
all of which can be done in one cycle. This mechanism is much more efficient and easier to 
implement than using known register files or rename buffers since the contents of the 
registers are not copied to the target thread. If creating a child thread requires copying the 
registers' contents, a thread having 32 registers would require a register file and/or rename 
buffer with 32 read and 32 write ports to do this in one cycle. 

The process of copying registers for creating a child thread is described with an 
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example. Figures 8 and 9 show K architectural state pointers for two threads and M number 

of reference counters, associated with a M-entry operand queue. To simplify the discussion, 

assume that the parent thread's architectural register values are to be copied to the child 

thread's registers. Since the operand file is shared by two threads, M should be some 

number greater thmi 2K. To copy all K registers from the parent thread to the child thread, 

the operand queue entry in each of the parent thread's architectural state pointers is copied to 

the corresponding architectural state pointer in the child thread. This can be done in one 

cycle without requiring much hardware since each pointer consists of a few bits. In addition, 

the reference count associated with each entry in the architectural state pointers is 

incremented by 1 . In the example shown in Figure 8, the parent thread's RO and Rl refer to 

the K^^ entry while Rk-1 refers to the 0*^ entry. The reference counts of the and 0^ entry 

are 2 and 1, respectively. After the register copy operation, the child thread's RO and Rl 

pointers refer to the entry and Rk-1 pointer refers to the 0* entry, same as the parent 
• th 

thread's pointers. The K reference count is now 4, since two more registers from the child 
thread reference the entry. Similarly, the 0* reference count is now 2. The newly 
created thread's future state pointers have the same values as the architectural state pointers 
since there are no outstanding instructions. 

The previous example on Figures and 8 and 9 assume that thread creation occurs when 
the thread-creating instruction is the next instruction to complete. A more eflBcient way to create 
a child thread and copy some or all of the parent thread's registers to corresponding child thread's 
registers is when the thread-creating instruction is at the decode stage. In this case tiierc may be 
outstanding instructions, which are the instructions that appear earlier in program order than the 
thread-creating instruction and that have not completed. To provide the precise exception model 
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of programming, the thread-creating instruction must execute after all of the outstanding 
instructions have completed without incurring an exceptioa When the thread-creating 
instruction is decoded, Ute instruction is stalled until it is determined that none of the outstanding 
instiiictions can cause an exception. Then, the contents of the parent tiiread's future state 
pointers, and not the architectural state pointers as in the previous example, are copied to the child 
thread's architectural and future state pointers. The parent tiiread's future state pointers may hold 
different values tiian its architecUual state pointers at tiiis point, due to outstanding instructions, 
but tiie two sets of tiie pointers will have the same values \^4ien all of tiie outstanding instiiictions 
are completed without incurring an exception. This schetne allows the child tiiread to be created 
earlier than waiting until the tiiread-creating instiuction is tiie next instiiiction to complete. 

When a thread is terminated, any shared resources assigned to it can be freed to 
improve resource utilization. Terminating a thread is similar to processing an exception 
condition, as now described. When a tiiread is to be terminated, any outstanding instructions 
must be cancelled, using the mechanisms discussed previously. Specifically, as each 
cancelled instruction is processed, tiie reference count associated witii tiie entry assigned to 
tiie destination register or an immediate operand is decremented by 1 . The architectural state 
pomters are not modified. When all outstandmg instructions have been processed, any 
operand queue entries tiie tiiread holds can be freed simply by decrementing the reference 
count associated witii each of tiie register's architectural state pointers. Those entries whose 
reference count becomes 0 naturally become free. 

Although the present invention has been described in accordance witii the 
embodiments shown, one of ordinary skill in tiie art will readily recognize tiiat tiiere could be 
variations to tiie embodiments and tiiose variations would be witiiin tiie spirit and scope of tiie 
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present invention. Accordingly, many modifications may be made by one of ordinaiy skill 
the art without departing from the spirit and scope of the appended claims. 
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